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In spiking neural networks an action potential could in principle trigger subsequent spikes in the 
neighbourhood of the initial neuron. A successful spike is that which trigger subsequent spikes 
giving rise to cascading behaviour within the system. In this study we introduce a metric to assess 
the success of spikes emitted by integrate-and-fire neurons arranged in complex topologies and 
whose collective behaviour is undergoing a phase transition that is identified by neuronal avalanches 
that become clusters of activation whose distribution of sizes can be approximated by a power-law. 
In numerical simulations we report that scale-free networks with the small-world property is the 
structure in which neurons possess more successful spikes. As well, we conclude both analytically 
and in numerical simulations that fully-connected networks are structures in which neurons perform 
worse. Additionally, we study how the small-world property affects spiking behaviour and its success 
in scale-free networks. 

PACS numbers: 05.65.+b, 05.70.Fli, 05.70.Jk 


I. INTRODUCTION 

Cascading behaviour in complex networks refers to a 
domino effect resulting from the activation of nodes in a 
network whose internal dynamics are subject to threshold 
mechanisms and propagation of events (e.g. action po¬ 
tentials, diseases, fads, articles becoming cited, etc.). In 
the context of cascading behaviour, the success of a node 
refers to its capacity (once it becomes active as a result of 
its internal dynamics) to trigger subsequent activations in 
its neighbourhood. Intuitively, network structure plays a 
major role on determining how activity spreads through 
a system. In this respect, the discovery of topological 
features such as the small-world property and the scale- 
invariance of the degree distribution in many real-world 
networks provided a new perspective in which to analyze 
cascading activity. 

The study of complex networks took off when it was 
observed that the essence of real-world networks cannot 
be captured by the random network model introduced 
by Erdos and Renyi [I] nor by regular structures such as 
lattices. The Watts and Strogatz model [5] was proposed 
to describe a class of networks that lie halfway between 
randomness and regularity. This class of networks are 
characterized by a small average shortest path length (a 
feature observed in random networks) and an average 
clustering coefficient significantly larger than expected 
by chance (a feature observed in regular lattices). Taken 
together, these properties offer a structural benefit to the 
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processes taking place within the network, such as opti¬ 
mal information transmission that results from speeding 
up the communication among otherwise distant nodes. 
A term that summarizes the presence of these two prop¬ 
erties is that of the small-world property. Networks that 
exhibit the small-world property are so diverse and can 
be found in social, technological and biological contexts, 
to name a few 3j. 

Scale-invariance in the distribution of the node degrees 
of a network is a phenomenon observed in real-world net¬ 
works. Networks that exhibit this particular feature are 
known as scale-free networks. In this type of networks 
the probability P{k) that a node connects to k other 
nodes follows a power-law P(k) ~ A: -7 [4]. It implies 
the existence of many poorly connected nodes coexist¬ 
ing with very few but not negligible massively connected 
nodes hubs. Scale-invariant degree distribution and the 
small-world property are by no means exclusive and in 
fact many scale-invariant networks are also small-world. 

When it comes to the dynamics of a system compris¬ 
ing numerous interconnected elements interacting non- 
linearly, a considerable number of studies have been ded¬ 
icated to the occurrence of power-law behaviour and its 
relationship to the notion of phase transitions, 1//“ noise 
and self-organized criticality (SOC) |5| that results from 
the collective dynamics of the threshold units comprising 
a system. 

The concept of SOC, has been suggested to explain the 
dynamics of phenomena as diverse as plate tectonics [6] , 
piles of granular matter [7j, forest fires [5], neuronal 
avalanches [9] (see below), and mass extinctions lOl . 
among several others. Moreover, SOC implies the ex¬ 
istence of a critical point that becomes an attractor in 
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the collective dynamics of a system. Such a critical point 
or regime denotes a state of the system in which the col¬ 
lective dynamics are undergoing a phase transition. As 
such, it represents the boundary between two different 
states of the system (e.g. order and chaos) and it is iden¬ 
tified by the presence of power laws in the distribution of 
events, the divergence of the correlation length, among 
others [5]. 

In the context of brain networks, the presence of neu¬ 
ronal avalanches has been observed as the result of spon¬ 
taneous activity in local field potentials of cultured slices 
of rat cortex m . and in the superficial cortical layers of 
awake, resting primates m- As the name suggests, neu¬ 
ronal avalanches are an example of cascading behaviour 
triggered by spiking in groups of neurons. The observed 
avalanches are stable and repeatable spatiotemporal pat¬ 
terns of activity BE which might relate them to memory 
mechanisms inside the brain. 

In models of neuronal avalanches, it has been reported 
that the distribution of their sizes as well as the distribu¬ 
tion of their durations can be approximated by a power 
law with very precise exponents in the thermodynamic 
limit as well as scaling relationships among system sizes 
and exponents ma. As mentioned earlier, this phe¬ 
nomenon has also been reported in real brain tissue (al¬ 
though with finite-size effects) [TT]. Power-law behaviour 
in the dynamics of neuronal avalanches relate this biolog¬ 
ical process to the notion of SOC described above. 

Critical dynamics of brain networks have been studied 
thoroughly in artificial models, and it has been found that 
the critical regime implies several computational benefits 
for the system, namely: optimal information transmis¬ 
sion and maximum dynamic range [lSj . maximum infor¬ 
mation storage mini, stability of information transmis¬ 
sion [18], among others. Hence the criticality hypothesis 
for brain dynamics, which states that neural networks op¬ 
erate at the edge of chaos , that is, at the critical point in 
a phase transition between total randomness and boring 
order [T9], 

In this paper we study the success of integrate-and-fire 
nodes in terms of their capacity to trigger subsequent 
spikes in their neighbourhood once they become active, 
and thus giving rise to cascading activity. We study this 
local performance in topologies such as fully-connected, 
random, and scale-free networks with varying amounts 
of the small-world property when the systems are at the 
critical state of their collective dynamics. 


II. MODEL 
A. The Eurich model 

The model consists of N non-leaky integrate-and- 
fire nodes and was formulated for fully-connected net¬ 
works [9]. This model exhibits critical dynamics in the 
collective behaviour of the units comprising the sys¬ 
tem. This fact is identified by the presence of neuronal 


avalanches, whose size and durations can be approxi¬ 
mated by a power law. In analytical examinations, the 
exponents derived for such distributions are 7 = —3/2 
and S = — 2 for avalanches sizes and durations respec¬ 
tively, in the thermodynamic limit of fully-connected net¬ 
works BM- Here we will extend such a model by con¬ 
sidering also heterogeneous directed networks. 

In the model, each node j is characterised by a contin¬ 
uous variable known as the membrane potential h, which 
is updated in discrete time according to the equation: 

N 

hj(t + 1 ) = hj(t) + AijWijSi(t ) + I ext ( 1 ) 

i —1 

where A denotes the asymmetric adjacency matrix with 
entries A^ = 1 if node i sends and edge to node j , and 
Aij = 0 otherwise; Wq denotes the synaptic strength 
from node i to node j; Si(t ) £ { 1 , 0 } represents the state 
of node i (active or quiescent, respectively) at time f; and 
I ext denotes external input which is supplied to a node 
depending on the state of the system at time t. This 
mechanism of external driving works as follows: if there 
is no activity at time t, then a node is chosen uniformly 
at random and its membrane potential is increased by a 
fixed amount through the variable I ex t- If hj(t) exceeds 
the threshold 0 , which in simulations is set to unity, then 
node i emits a spike, which changes the state of this node 
to active ( Si(t ) = 1 ) and propagates its activity through 
its synaptic output. Afterwards, the node is reset, ie. 
hi(t + 1 ) = 0 . 

The coupling strength Wij for every node i sending 
an edge to node j is set according to the equation 
= a/{e) where a is the control parameter of the 
model and (e) denotes the mean degree of the network; 
which is the same for all the network structures consid¬ 
ered (see below), except for fully-connected networks, in 
which case mean connectivity is (N — 1). 

In the first stages of our experiments we let the pa¬ 
rameter a take values in the interval ( 0 , 1 ), and then we 
measure the deviation from the best power-law fit to the 
distribution of avalanche sizes (see Sect. [HD] for details). 
The values of a for which the deviation is at its minimum 
are those who lead the system to the critical state, and 
define the critical interval. In a second stage of our exper¬ 
iments we re-start the system considering only values of 
a taking uniformly at random from the critical interval. 
The critical interval varies for different types of networks 
and system sizes. 

As mentioned above, in this work we consider different 
types of network structures as well as system sizes. The 
topologies considered are: 

i. fully connected, 

ii. random, 

in. scale-free with low mean clustering coefficient (CC) 
and power-law in the out-degree distribution, 
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iv. scale-free with high mean CC and power-law in out- 
degree distribution, 

v. scale-free with low mean CC and power-law in in¬ 
degree distribution, 

vi. scale-free with high mean CC and power-law in in¬ 
degree distribution. 

System sizes for each of these classes of networks are: 
128, 256, 512, and 1,024. We should point out an impor¬ 
tant aspect of the networks that we consider. For the case 
of random and scale-free networks the number of edges is 
the same for each system size, which results in the same 
average connectivity for these types of networks. Thus, 
the topology results from a particular permutation of the 
edges. However, this edge permutation is not arbitrary, 
but results from a particular algorithm depending on the 
structure that we want to obtain. In the case of random 
networks the mechanism, by which we permutate such 
edges, is given by the Erdos-Renyi model pH], whereas 
for scale-free networks with tuneable clustering we follow 
the ideas in Ref. |21l , in which the authors present an ex¬ 
tended version of the BA model H] , in which a large mean 
clustering coefficient is achieved for scale-free networks 
by adding a triangle-formation step to the preferential 
attachment algorithm. This algorithm produces scale- 
free networks whose mean clustering coefficients match 
better the observations in real-world networks. 

In the following section we describe the structural dif¬ 
ferences of networks whose in-degree distribution follows 
a power-law, and networks whose out-degree distribution 
follows a power-law. 


B. Broadcasting hubs and absorbing hubs 

Scale-free networks are characterized by a power-law 
approximation of their degree distribution |4]. In the 
case of directed networks, there are two degree distribu¬ 
tions, one corresponding to the out-degrees of nodes and 
another one for the in-degree of nodes. In general, real- 
world networks are directed, and very often both their 
degree distributions can be approximated by a power-law 
(e.g. the World Wide Web HOI), or at least one of them 
(e.g. citation networks [20]). In either case, the presence 
of a long-tail in the out-degree distribution of a network 
implies the existence of broadcasting hubs, that is, nodes 
that have massive outgoing connections compared with 
other nodes in the system. On the contrary, the presence 
of a long-tail in the in-degree distribution implies the ex¬ 
istence of absorbing hubs. Here, we are interested in an¬ 
alyzing how collective dynamics develop for the case of 
networks with broadcasting hubs and for networks with 
absorbing hubs. In the following, scale-free networks with 
absorbing hubs will be labeled as in-degree scale-free net¬ 
works , whereas those that contain broadcasting hubs will 
be termed out-degree scale-free networks. 

As mentioned in the introduction, the small-world 
property is not a binary one, and as such, there exist 


degrees of what we would call small-world-ness. All the 
scale-free networks considered possess the small-world 
property up to a certain amount. In our model we con¬ 
sider two levels of mean clustering for scale-free networks 
(low and high) by tuning a simple parameter [2T] . The 
process of tuning the mean clustering coefficient in these 
types of networks has an immediate effect on the de¬ 
gree of small-world-ness of such networks. Scale-free net¬ 
works with low mean clustering coefficient possess a low 
degree of small-world-ness when compared against scale- 
free networks with high mean clustering coefficient. In 
this study, we inquire on the effects on criticality for dif¬ 
ferent degrees of small-world-ness. 


C. Node success 

We introduce a local measure of the performance of a 
node during simulation time. The node success of node 
i at time t is the fraction of out-neighbors of this node 
that become active at time t + 1 when node i spikes at 
time t , in other words: 


Vi(t) 


XqLi + 1) 



( 2 ) 


where A is the adjacency matrix, and Sj(t + 1) the state 
of node j at time t + 1 . 

Thus, node success measures the performance of an 
individual spike in terms of the subsequent spikes trig¬ 
gered by such initial activation, which occur within the 
out-neighborhood of a given node. In contrast to many 
other popular network statistics (e.g. degree distribu¬ 
tion, branching ratio m, etc.) node success is a local 
measure of performance. 

We consider two different averages of this measure. 
First, the mean node success per node which results from 
considering only the times in which a node spikes and 
then averaging its node success at each of these times. 
Second, the mean node success per time step which re¬ 
sults from averaging the node successes of all nodes in 
the system at a particular time step. 


D. Numerical implementation 

When starting simulations, all membrane potentials 
are initialised at random taking values in the interval 
(0,1), whereas all states are set to inactive. By means of 
external driving, activity inside the system in the form 
of neuronal avalanches is guaranteed to occur. However, 
avalanche sizes and their durations will not always be the 
same nor can they be predicted. 

Both the relaxation time towards the critical state as 
well as the sampling time needed to assess criticality de¬ 
pend on the system size. For networks consisting of 128 
nodes we allow critical dynamics to set in for one million 
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time steps according to the Eurich model ©]; for net¬ 
works of 256 nodes we allow critical dynamics to set in 
for two million time steps, for networks comprising 512 
we allow for three million time steps, and finally for net¬ 
works of 1, 024 elements the dynamics run for four mil¬ 
lion time steps. This selection of times is appropriate for 
large events (that is avalanches that extend to the whole 
network) to take place during simulation time. With 
this in mind, we expect to have small events (i.e. small 
avalanches) coexisting with large events (i.e. avalanches 
that span the whole system). An inspection of the dis¬ 
tribution of avalanche sizes after this driving stage shows 
a distribution that can be approximated by a power law 
with a cut- off due to the finite nature of the system (see 
Sect. Ill A). The power-law approximation of such a dis¬ 
tribution implies that the system is in the critical regime 
with very frequent small events coexisting with rare but 
not negligible large events. 

We assess the quality of such a power law through 
the mean-squared deviation Ay from the best-matching 
power law with exponent 7 obtained through regression 
in log-log scales. Our choice of using this method is due 
to its simplicity and justified by the asymptotic unbiased¬ 
ness of the estimation. When this error function is at its 
minimum, that is, when the data is best approximated 
by a power-law distribution with exponent 7 , is when the 
system is at the critical state. 

For our experiments we consider 50 different networks 
per class (u to vi described above) and system size 
for the sake of statistical robustness. In the case of 
fully-connected networks, as there exists only one fully- 
connected network of size N , randomness is introduced 
in the seed of the pseudorandom number generator used 
in our code for each realization of the experiment rather 
than in the structure as for the other network classes 
considered. Experiments were carried out in the EDDIE 
computer cluster of the University of Edinburgh. 


III. RESULTS 



(a) Power-law fitting deviation per time step 



(b) Log-log plot of distribution of avalanche sizes S 


FIG. 1: Deviation from power-law matching per time 
step for a network of size N = 512. The minimum error 
reached at time step 3 x 10 6 corresponds to the critical 
regime identified by a power law distribution of 
avalanche sizes. We show the averaged value of all 


realizations, but we do not present error bars in Fig. lb 
in order to make its presentation more accessible. 


A. Avalanche size distribution follows a power-law 


As mentioned in Sect. |IID[ we assess the quality of the 
power-law approximation to the distribution of avalanche 
sizes by estimating the deviation from the best power- 
law fit. When such an error function reaches a mini¬ 
mum value of less than or equal to 0.05, we consider the 
event-size distribution as well approximated by a power- 
law and conclude that the system is in a critical state. 


Fig. la shows the power-law fitting error as a function of 
simulation time for the distribution of avalanche sizes for 
scale-free and random networks of size N = 512. This 
figure shows the deviation, Ay, of our data from the best 
matching power law with exponent 7 . In this figure, we 
present mean values and standard deviations of Ay ob¬ 
tained from the realizations of our experiments. Fig. |lb| 
shows the distribution of avalanches sizes for all scale-free 


and random networks of size N = 512 at criticality (ie. 
when Ay < 0.05 around time step 3 x 10 6 in Fig. [la| . 
We show the averaged value of all realisations, but we 
do not present error bars in Fig. |lb| in order to make its 
presentation more accessible. Although we show the dis¬ 
tribution of avalanche sizes and the deviation from the 
best matching power-law for a particular system size, all 
system sizes exhibit a similar behaviour. 

Moreover, following Ref. [22] we inspect the value of 
the largest eigenvalue of the matrices W associated to 
each network and whose entries w. i:j denote the synaptic 
weight between node i and j. The authors in Ref. [H] 
observe that the largest eigenvalue of the weight matrix 
governs the dynamics of the system. Through an ana¬ 
lytical examination, it is reported that when the largest 
eigenvalue equals unity the system is at the critical state. 
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Type 

Subtype 

Size 

A 


Low Mean CC 

128 

0.906 ±0.029 



256 

0.9 ±0.02 

Out-degree scale-free 


512 

0.95 ±0.01 



1,024 

0.97 ±0.008 


High Mean CC 

128 

0.89 ±0.04 



256 

0.91 ±0.03 



512 

0.91 ±0.01 



1,024 

0.94 ± 0.01 


Low Mean CC 

128 

0.98 ±0.02 



256 

0.99 ±0.01 

In-degree scale-free 


512 

1.0006 ±0.006 



1,024 

1.001 ± 0.004 


High Mean CC 

128 

0.96 ±0.02 



256 

0.99 ±0.02 



512 

1.002 ±0.014 



1,024 

1.01 ±0.017 

Random 


128 

0.92 ±0.012 



256 

0.99 ± 0.022 



512 

0.97 ±0.001 



1,024 

0.98 ±0.005 

Fully-connected 


128 

0.91 ± 0.034 



256 

0.93 ± 0.0006 



512 

0.95 ±0.001 



1,024 

0.98 ± 0.0005 


TABLE I: Largest eigenvalue A of matrix W of synaptic 
weights. It has been found analytically that A = 1 is 
associated with a system at criticality [33] . The 
synaptic weight matrices of our networks have A « 1 
due to finite-size effect. (We present mean values and 
standard deviations.) 


9000 



system size 


(a) Total number of spikes 



system size 


(b) Total mean node success 

FIG. 2: Average number of spikes emitted by 
fully-connected networks per system size and their 
mean success. (Error bars denote standard deviations.) 


In Table [T] we report the value of the largest eigenvalue 
A of the weight matrices associated to our networks. In 
our experiments the critical state is not only identified 
by the power-law distribution of avalanche sizes (Fig. [l]) 
but also by the value close to unity of A. Due to finite 
size effects this value is not exactly unity but close to it. 


B. Small-world property boosts network activity 


The small-world property affects the rate of firing of 
nodes comprising a network. Fully-connected networks 
are structures in which all nodes exhibit a similar fir¬ 
ing rate, giving rise to a well defined mean and vari¬ 
ance (see Fig. 2a) unlike scale-free networks in which the 
variance of the firing rate seemingly diverges, and thus 
its mean cannot characterize the network activity (not 
shown here). In fact, this latter type of structure con¬ 
tains nodes whose firing rate can far exceed the firing 


rate in fully-connected networks (see Fig. 3a). 

Fig. [2a] shows the average of the total number of spikes 
over all nodes in fully-connected networks for all system 
sizes considered. In contrast, we will show that nodes 
in heterogeneous topologies can perform better; in this 
case, scale-free networks possess nodes with higher firing 
rates than random networks. It is worth mentioning that 
this behaviour is verified in all system sizes considered. 

We pose several questions regarding the relationship 
between network structure and dynamics. The first is: 
are the nodes with higher local CC those that spike more 
often, that is, do better-clustered nodes fire more? Sur¬ 
prisingly, nodes with low local CC exhibit a larger spiking 
rate than more clustered nodes. Fig. [3a] shows this be¬ 
haviour for networks of size N = 1,024. Here we show 
not only that low locally clustered nodes fire more but 
also that in in-degree scale-free networks nodes can fire 
more than in any other type of structure. 

A question that arises at this point is the following: are 
those low locally clustered nodes who spike so frequently 
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local CC 


(a) Total number of spikes 



degree 

(a) Low mean CC 



local CC 




(b) Total mean node success 


(b) High mean CC 


FIG. 3: Total number of spikes and mean node success 
per local CC for heterogeneous topologies of size 
N = 1, 024. Low clustered nodes are responsible for- 
spiking more frequent in scale-free networks. Their 
spikes tend to be more successful as well. 


FIG. 4: In scale-free networks well connected nodes are 
less clustered. Here we show for size N = 1,024 and 
in-degree scale-free nets with the two levels of mean CC 
(low and high). For the case of out-degree scale-free 
nets, degree distributions are switched so that 
out-degree becomes in-degree and vice versa. 


in in-degree scale-free networks the absorbing hubs or any 
other type of node? Taking a look at network topology, 
we verify that indeed hubs (either absorbers or broad¬ 
casters) are in general low locally clustered. In Fig. [4] 
we present for scale-free networks of size N = 1,024 and 
two levels of mean CC (low and high) the relationship 
between in-degree/out-degree and local CC. The more a 
node is in-connected the lower its local CC (likewise when 
reversing the direction of edges, which yields broadcast¬ 
ing hubs). 

Then, we verify that better-connected nodes possess 
higher firing rate than any other type of node. In Fig. [5] 
we present how the two different degree distributions (in 
and out) are related to spiking activity in scale-free net¬ 
works. Fig. 5a shows this for low mean clustered scale- 
free networks, whereas Fig. |5b| shows it for high mean 
clustered networks. For the case of out-degree scale-free 
networks, that is, networks that possess broadcasting 


hubs we do not observe any correlation between node 
out-degree and firing activity. This occurs in out-degree 
scale-free networks with low and high mean CC, and 
across all system sizes. However, for the case of in-degree 
scale-free networks, that is, networks that include ab¬ 
sorbing hubs, we observe a positive correlation between 
node in-degree and spiking. This behaviour occurs in all 
system sizes. Interestingly, the behaviour of in-degree 
scale-free networks with low mean CC differ from the 
behaviour of the same type of network with high mean 
CC. Both exhibit a positive correlation between in-degree 
and total number of spikes, nevertheless low mean clus¬ 
tered networks exhibit a linear trend, whereas high mean 
clustered ones exhibit a non-linear trend. This suggests 
that as a network becomes more clustered (and more 
small-worldly) the activity of their nodes exhibit a more 
quadratic dependency of the node’s in-degree. We do not 























7 



outdegrce-SF (outdegree) A 
outdegree-SF (indegree) x 
indegree-SF (outdegree) 


indegree-SF (indegree) □ 
y = 255.76X + 29228.. 

A 


<*•$.* 


° #o ° .-13 LI 15 

J§L n UPd ^ 013 m 

Stt „ „ B □ 


° a " 

.-.Sir* 




,, L ,,,, 


0 20 40 60 80 100 120 140 160 180 200 


degree 


(a) Low mean CC 


180000 

160000 

140000 

120000 

100000 

80000 

60000 

40000 

20000 

0 

0 20 40 60 80 100 120 140 160 180 200 



(b) High mean CC 

FIG. 5: In in-degree scale-free nets the absorbing hubs 
spike more than any other type of node and their 
in-degree is correlated with the amount of spikes fired. 
For networks with higher mean CC this correlation is 
quadratic. We show this behaviour for a network of size 
N = 1,024. 


explore this hypothesis in the present work, but it is the 
direction of future research. 

At first, the observation of a positive correlation be¬ 
tween spiking and node in-degree would seem very obvi¬ 
ous, in the sense that nodes with high in-degree are driven 
beyond threshold frequently by the action of their in¬ 
neighbors. However a high in-degree cannot explain com¬ 
pletely the high firing rate of this type of nodes, because 
this would also predict that nodes in fully-connected net¬ 
works would possess a high firing rate, which is not the 
case (see Fig. 2a). What is happening in fully-connected 
networks that prevents nodes from firing as much as 
the other heterogeneous structures considered even when 
these nodes are massively connected? We suggest that 
an obstructing behaviour is occurring in this globally- 
coupled structure. We name this phenomenon spike jam¬ 


ming and we will discuss it in detail in Sect. HID 


Random networks behave somehow similarly to scale- 
free networks in the sense that there is a positive correla¬ 
tion between node in-degree and spiking, but not between 
this latter and node out-degree. However, this type of 
structure does not reach the same amount of spiking per 
node as scale-free networks due to the random nature of 
their connectivity (not shown here). Thus, the presence 
of hubs account for the high firing rate in heterogeneous 
structures. 

Another question that arises at this point is the follow¬ 
ing. Does a high in-degree in scale-free networks account 
by itself for a high spiking activity? Or is it the joint 
action of in- and out-degree that explain this particular 
behaviour? In other words, could this high firing rate 
be explained by a specific configuration of in- and out- 
degree? To explore this question we considered the ratio 
of in-degree to out-degree per node, which for a node i is 
given by: 




Pi = 


ki 


A. 

2^ i=i 


(3) 


where the numerator is the in-degree of node i, and the 
denominator is its out-degree. The quantity p is equal 
to unity when a node has the same number of incoming 
and outgoing connections. In fully-connected networks 
all nodes possess this property. If p, > 1, then the in¬ 
degree of node i is larger than its out-degree, and Pi < 1 
when the opposite occurs. 


From fully-connected networks we have learned that 
homogeneity in node degree, that is, p = 1 for every 
node, is not a property suitable for spiking. Moreover, we 
verify this fact in heterogeneous structures where nodes 


with p 1 fire more than any other nodes. Fig. 6a 


shows this particular behaviour. There we present how 
spiking is improved as p grows larger than unity. The 
solid black line at left of Fig. [6a] marks the point where 
p = 1, ie. where in-degree equals out-degree. For all 
heterogeneous structures considered (excluding random 
networks) a larger in-degree than out-degree correlates 
with higher firing rate. 


In summary, in fully-connected networks nodes fire less 
than in any other topology. This is explained by the ho¬ 
mogeneity of the nodes comprising the network, which 
give rise to the phenomenon of spike jamming. In het¬ 
erogeneous topologies, scale-free networks fire more than 
random networks, and the firing activity is improved by 
the presence of absorbing hubs and high mean CC, which 
implies a larger degree of small-world-ness. However, for 
this to happen absorbing hubs must possess the right 
amount of outgoing connections which is represented by 
1 . 
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(a) Total number of spikes 



(b) Total mean node success 

FIG. 6: Total number of spikes and their success per 
ratio in-degree/out-degree for heterogeneous nets of size 
N = 1,024. The solid line at the left lies at p = 1 where 
in-degree equals out-degree. An equal number of 
incoming and outgoing connections cannot account for 
higher spike rate and spike success. 


C. Scale-free topologies comprise more successful 
nodes 

Spiking is not all that matters, since we should also 
consider the fate of a spike that has just been emitted. 
Here, we consider a successful spike one that triggers sub¬ 
sequent spikes from the nodes in the out-neighborhood 
of the node where the initial spike originated. As we are 
interested in the propagation of activity within the sys¬ 
tem, we would like to observe the sustained activation of 
nodes in subsequent time steps. This is where the notion 
of the branching ratio comes to hand. The branching 
ratio a is defined as the ratio of descendants that be¬ 
come active at time t +1 to ancestors that were active at 
time t. The quantity a has been used to characterize the 
critical state of a system m and to identify the regimes 
surrounding such a state. When a < 1 the system is sub- 


critical and activity dies out quickly, when this value is 
above unity, the system is supercritical and activity gets 
amplified pathologically at each time step. In between 
these two states lies the critical state in which activity is 
sustained until finite-size effects take place, during this 
regime a is equal to unity for a prolonged period of time. 

Recall that in Sect. |II C'| we defined the success of any 
give node i as the fraction of out-neighbors that become 
active at time t + 1 when node i spiked at time t. This 
quantity is similar to the branching ratio a in the sense 
that it estimates the amount of activity sustained in sub¬ 
sequent time steps. However, unlike cr our measure of 
node success is a local estimation of performance, which 
has more natural implications in the context of neuronal 
networks, in which a neuron does not have access to 
global metrics regarding the structure of the network. 

As mentioned above, spiking does not imply success, 
and node success as defined above is intimately related to 
the notion of criticality. Here we repeat the same ques¬ 
tions that we considered in the previous section, namely, 
how successful are nodes with high local CC? how suc¬ 
cessful are hubs? and finally, how network structure af¬ 
fect node success? 

For scale-free networks, nodes with low local CC are 
the most successful nodes. This behaviour is more evi¬ 
dent in in-degree scale-free networks with high mean CC 
(see Fig. 3b). These low local CC nodes are the hubs, 
however unlike the firing activity, mean node success per 
node does not exhibit a very clear correlation between in- 
or out-degree and success (see Fig. 6b). Finally, random 
networks do not exhibit any particular pattern regarding 
the success of their nodes. 

Which is the most successful topology? In other words, 
what is the structure that maximises the success per 
node? To answer this question we estimated the mean 
node success of the system per time step (see Sect. IIC) 
for all the topologies considered. 

For all system sizes we observe that fully-connected 
networks are the type of structure that performs worst 
(see Fig. 2b), followed by random networks. For the case 
of scale-free networks, in-degree scale-free networks with 
high mean CC are the most successful topologies. Second 
in place are in-degree scale-free networks with low mean 
CC, followed by out-degree scale-free networks with high 
and low mean CC, respectively. Thus, absorbing hubs 
in a scale-free structure allow for more node success per 
node if accompanied by a high degree of small-world-ness. 

Recall that the scale-free and random topologies have 
the same number of edges. Therefore, in-degree scale- 
free-ness with high mean CC is the permutation of edges 
that maximises node success. Fig.[7]shows the aforemen¬ 
tioned behaviour for networks of size N = 1,024, the 
same is observed in all the other system sizes considered. 

Moreover, the value of the mean node success per time 
step is upper bounded. For a system to remain in the crit¬ 
ical regime the mean node success must remain below a 
certain value. In other words, a high mean node success 
is related to the supercritical regime in which nodes fire 
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FIG. 7: Mean node success for scale-free and random 
networks of size TV = 1,024. In-degree scale-free 
networks with high mean CC possess more successful 
nodes than any other topology considered. (Error bars 
denote standard deviations.) 


where li denotes the set of in-neighbors of i, and recall 
that Wij describes the coupling between nodes i and j, 
and that Sj(t) = 1 if node j spikes at t and 0 otherwise, 

i.e. 


Sj(t) 


1 if hj(t) > 9 
0 otherwise. 


(5) 


Note that the membrane potentials are not driven dur¬ 
ing the avalanche. The avalanche ends when there are no 
more spiking nodes. Afterwards, the membrane poten¬ 
tials are then driven again each time step until another 
avalanche is triggered. This process is repeated until the 
simulation ends. 

In a fully connected network each node has (TV — 1) 
out-neighbors, where TV denotes the number of nodes in 
the network. Specifically, the out-neighbors of node i are 
all the nodes j such that j ^ i. Recall that the node 
success of node i at time step t is defined as the 
fraction of out-neighbors of i which spike at time step 
t + 1, given that i spikes at t. Therefore, if i spikes at 
f[23l. Eqn. (J2| becomes 


constantly (not shown here); so that, if we were inter¬ 
ested in maximizing the value of the mean node success 
we would have to leave the critical regime. Additionally, 
we report that the upper bound of the total mean node 
success at criticality decreases as the system size grows. 
An example of this is shown in Fig. |2b| where the maxi¬ 
mum value of node success for fully-connected topologies 
decrease with system size. Although not shown here, this 
phenomenon is observed in all other topologies and sys¬ 
tem sizes. 


D. Upper bound of mean node success for 
fully-connected nets 

As mentioned in the previous section, fully-connected 
networks perform worst as measured by the mean node 
success per time step. In this section we derive an ana¬ 
lytical expression for the upper bound of this metric for 
globally-coupled structures. 

Recall that hi(t) denotes the membrane potential of 
node i at time step t, and that 6 > 0 denotes the thresh¬ 
old membrane potential required to trigger a spike in a 
node: node i will spike at t when Ti,(f) >6. At t = 0 
the membrane potentials take values /q(0) < 9 for all 
i. These potentials are then driven externally each time 
step until the membrane potential of one node is taken 
above the threshold membrane potential - triggering an 
avalanche. During an avalanche the membrane potentials 
evolve as follows: 


hi(t + 1 ) 


0 if hi(t) > 9 

h i{t) + w ij s j(t) otherwise, (4) 

jeii 


Vi{t) 


S(t + 1) 
TV- 1 


( 6 ) 


in a fully connected network, where S(t) denotes the 
number of nodes which spike at time step t. Note that the 
right-hand side of the above expression is independent of 
i. This reflects the fact that tpi(t) is identical for all nodes 
i which spike at t. For this reason we will henceforth omit 
the subscript i, and deal only with the quantity the 
node success of any node which spikes at t. This obser¬ 
vation is only valid for fully-connected networks in which 
all nodes receive connections from each other and send 
connections likewise. 

Consider the mean node success for nodes which spike 
during any period of r time steps. Since S(t) nodes spike 
at f, this is given by 


M = ^J2S(t)ip{t), (7) 

° t =l 

where without loss of generality we have chosen the r 
timsteps to be t = 1, 2 ,..., r, and 


S = j2 s (t) ( 8 ) 

t— l 

is the total number of spikes which occur during this 
period. A crucial aspect of the dynamics described above 
is that hi(t + 1) = 0 if node i spikes at time t. It is there¬ 
fore impossible for node i to spike on two adjacent time 
steps - after spiking, the membrane potential of node i 
is frozen (due to refractoriness) to be zero for a single 
time step, during which time it cannot ‘accumulate ac¬ 
tivity’ from spiking in-neighbors in the manner described 
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by Eqn. Q. This constraint can be expressed mathemat¬ 
ically as 


Si(t) + Si{t + 1) < 1 for all i, t, (9) 

and leads to the following theorem: 

Theorem 1. (ip) has an upper bound of 

N , , 

vP) max 2(N _ 1 ) ' 

in the limit r —> oo. This upper bound is realised when 
S(t) = N/2 for all t > 1. 

Proof: Substituting Eqn. ^ into Eqn. (J7| gives 

W = s(rhi)£; sms{t + 1) - (11) 

Defining the quantity 

5(f) = S(t)/VS, (12) 

Eqn. can be expressed as 

(<p) = (jTi)E^ (t + 1 )' ( 13 ) 

Now, taking the summation of Eqn. ([£]) over all nodes i 
yields [24] 


S(t) + S(t + 1)<N for all t (14) 

after noting that 


S(t) = £>(*). 


(15) 


Subtracting S(t + 1) from both sides of Eqn. (141 and 
then multiplying throughout by 5(t + 1) yields 

S(t)S(t + 1) < S(t + 1) [N - S(t + 1)] for all t, (16) 

which can be expressed as 


5(t)5(t + 1) < 5(f + 1 ) 


N 

7s- s(t+1) 


for all t (17) 


after dividing both sides by S. [25] The right-hand side 
of the above inequality, which we denote as 


£(t + 1) = 5(t + 1) 


N 

7s- S{t + 1) 


(18) 


is hence an upper bound for term t on the right-hand side 
of Eqn. (13). Therefore we can write 


1 


(FVT)E« f + 1 >- 


(19) 


Now, £(t + 1) is maximised when 5(f + 1) = N/ (2 \/S). 
Therefore S(t + 1) = N / (2\/S) for t = 1,2,..., r pro¬ 
vides an upper bound for (ip), which, from substituting 


the aforementioned S(t- 1-1) into into Eqns. (18) and (19), 
can be shown to be 


(^)max — 


1 


f-V- 

(N-l)\2j S 


( 20 ) 


However, from Eqn. (12), S(t + 1) = N/(2\/S) for 
t = 1, 2,..., r corresponds to S(t + 1) = N/2 for t = 
1,2 ,,t, and hence from Eqn. ([ 8 ]) also corresponds to 


S = S(l) + J2 S (t) = S(1) + (t~1)N/2. (21) 

t =2 


Substituting this into Eqn. (20) gives 

2 


(^)max — 


1 


Y 


N 


(N- 1) V 2 ) 5(1) + (t — l)JV/2’ 


( 22 ) 


We emphasise that this is realised when S(t + 1) = N/2 
for t = 1, 2,..., t , or equivalently, when S(t) = N/2 for 
t = 2, 3,... , t + 1. Noting that 5(1) cannot exceed N, 
Theorem Q] results when the limit r —> oo is taken. ■ 


Some remarks are due with regards to Theorem |T| 
Firstly, (ip) as r —> oo, which we henceforth refer to sim¬ 
ply as (ip ), describes the mean node success over all nodes 
over all time. 

Secondly, Theorem [T] applies in a very general way 
to fully-connected networks, in the sense that for the 
purpose of proving the theorem we have made no as¬ 
sumptions regarding how the system is driven between 
avalanches, or the initial values of the membrane poten¬ 
tials. Furthermore, we have made no assumptions re¬ 
garding the specific values of 6 or wtj. 

Thirdly, since Theorem [T] pertains to a fully-connected 
network, it follows that any network with (ip) > (<£>) max 
cannot be a fully connected network. In a similar vein, 
if one wishes to construct a network with (ip) > (</?) max 
starting from a fully connected network, it is necessary 
that some connections between nodes are removed, that 
is, such a network should part from a massively connected 
to a less connected structure. 

Fourthly, (yj) m ax decreases monotonically with N, and 
(<p)max —> 1/2 in the thermodynamic limit, i.e., N — > oo. 

Finally, and most importantly, the theorem is non¬ 
trivial in the sense that one can easily conceive of net¬ 
works of size N whose global node successes can poten¬ 
tially exceed (v?) m ax- For instance, consider the network 
corresponding to a ‘directed ring’, where A 12 = 1 , A 23 = 
1 ,..., j4(iv_i)JV = 1 , Ajsri = 1 , and A l: j = 0 for all other 
elements of the adjacency matrix. If Wij = 6 , then as¬ 
suming, without loss of generality, that first spike in the 
network occurs on time step t = 1 at node 1 , then the 
spike propagates around the ring indefinitely: at t = 2, 
node 2 spikes; at t = 3 node 3 spikes, at t = N node N 
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spikes, at t = N + 1 node 1 spikes, etc. In this case it 
is easy to see that (ip) = 1, which is greater than (v?) m ax 
for TV > 2. Therefore the existence of an upper bound 
for fully-connected networks steins from some particu¬ 
lar property of their topology in combination with the 
dynamics described in Sect. [IT| 

What is this property of fully-connected networks 
which places this upper bound on their node success? As 
alluded to earlier, it is the fact that nodes are frozen for 
the time step after they spike which gives rise to the up¬ 
per bound in fully-connected networks. This behaviour 
gives rise to the phenomenon of spike jamming that we 
mentioned in Sect. |III B[ and which we describe in detail 
below. 

Consider a single node i firing at time step t in a fully- 
connected network. For this node to be maximally suc¬ 
cessful, it must trigger all TV — 1 of its out-neighbors, i.e., 
all other nodes in the network, to spike at t + 1. Suppose 
this happens, in which case ip, t (t) = 1. Consider now 
one of the nodes j ^ i which spikes at t + 1. For j to 
be maximally successful, all other nodes in the network 
must spike at t + 2. However, on account of refractori¬ 
ness, this is impossible. To elaborate, at t + 1, all nodes 
except for i, and including j, are spiking. Therefore all 
these nodes must be frozen at t + 2 - they cannot spike at 
t+ 2. On the other hand i, which spiked at t , while frozen 
at t + 1, is free to spike at t + 2. Hence, at best, only 
one of the TV — 1 out-neighbors of j. namely i. can spike 
at t + 2, and therefore at best <pj(t + 1) = 1/(TV — 1). 
For large TV, j is clearly very unsuccessful. The same 
applies for all other nodes which fire at t + 1. Hence 
the result is that, while i is maximally successful, the 
remaining TV — 1 nodes are extremely unsuccessful, and 
hence on average the whole network is unsuccessful dur¬ 
ing this avalanche - which we assume ends at t + 2. This 
example illustrates the effect which underpins the upper 
bound for fully-connected networks: if a node i spikes 
synchronously with one of its out-neighbors, then that 
out-neighbor is frozen for the next time step, and hence 
cannot spike on the time step after i spikes, which cur¬ 
tails the potential node success of i, and correspondingly 
the propagation of spikes throughout the network. Hence 
we refer to this effect as spike jamming. Note that afore¬ 
mentioned effect occurs in all networks, not just fully- 
connected networks. However, fully-connected networks 
are special in that all nodes are out-neighbors of each 
other, and hence this effect has more potential to curtail 
the node success in fully-connected networks than in any 
other network. 


IV. DISCUSSION 

In this paper we have presented arguments regarding 
the poor performance (in terms of spiking of individ¬ 
ual nodes and their success) of fully-connected networks 
at criticality showing at the same time that scale-free 
networks perform much better than any other topology 


when paired with the small-world property. 

Given that the heterogeneous topologies possess ex¬ 
actly the same number of edges, we conclude that scale- 
free-ness with high degree of small-world-ness is a permu¬ 
tation of edges that allows nodes to be more successful 
and to be more active in terms of the number of spikes 
emitted. In particular, we have verified the statement 
above for the case of in-degree scale-free networks, which 
feature the presence of absorbing hubs. However, real- 
world networks often comprise a more complex ecosystem 
in which absorbing hubs and broadcasting hubs coexist 
in the same network adding another layer of complexity 
to the dynamics within the system. Moreover, it is of¬ 
ten the case that the structure of real-world networks is 
not static, but they possess mechanisms by which nodes 
become connected and disconnected over time as well as 
network growth or shrinkage; features that affect the col¬ 
lective activity in ways that cannot be predicted with the 
current model. 

This leads us to the next consideration. What real sys¬ 
tem are we describing with the current model? From a 
certain point of view, the model used here is very lim¬ 
ited or simplistic, however a model of integrate-and-fire 
units can actually be a simplified model of many phe¬ 
nomena in nature. A model of threshold units that ac¬ 
cumulate activity from their vicinity and then propagate 
it when going beyond threshold can be used in princi¬ 
ple to model the spread of epidemics, piles of granular 
matter, the release of energy and relaxation of tectonic 
plates, the effects of a stock market crash, and the ac¬ 
tivity of neurons of the brain, among others. Thus, we 
believe that our model has a broad range of applications 
in diverse contexts, in which the presence of particular 
network properties such as the small-world property and 
long-tailed degree distributions have immediate effects on 
the dynamics of the system, be it the spread of a disease 
in a population, the propagation of stimuli on cortical 
networks, or the spread of rumors and fads within a so¬ 
cial network. Moreover, we believe that the introduction 
of the analysis of the success of a spike can be applied 
to the situations mentioned above. In other contexts, a 
spike could be thought of the transmission of an infec¬ 
tion among contacts, the death of a species in models 
of ecosystems, the failure of a power generator in power 
networks, and even in on-line social networks such as 
Facebook or Twitter we might regard a spike as the ac¬ 
tion of writing a post or a tweet In all these contexts, the 
fate of a spike is as relevant to the collective dynamics 
as is the network topology. Here we have shown that the 
combination of individual dynamics of nodes and topol¬ 
ogy determine the success of the spikes that spread across 
the system. 
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</ 2 i(f) is undefined if node i does not spike at t. 

In obtaining Eqn. (141 we use the fact that if x < a and 
y < b then x + y < a + b. 

In obtaining these equations we have used the following 
rules for manipulating inequalities: if x < a then x + c < 
a + c for all c; ii x < a then xc < ac if c > 0; if x < a 
then x/c < a/c if c > 0. In applying the second of these 
rules we used c = s(t + 1), which is > 0. In applying the 
last of these rules we used c = VS, which is > 0. 




